{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction and control tasks\n",
    "\n",
    "In reinforcement learning, we perform two important tasks, and they are:\n",
    "* The prediction task\n",
    "* The control task\n",
    "\n",
    "## Prediction task\n",
    "In the prediction task, a policy π is given as an input and we try to predict the value\n",
    "function or Q function using the given policy. But what is the use of doing this? Our\n",
    "goal is to evaluate the given policy.That is, we need to determine whether the given policy is good or bad.  How can we determine that? If the agent obtains\n",
    "a good return using the given policy then we can say that our policy is good. Thus,\n",
    "to evaluate the given policy, we need to understand what the return the agent would\n",
    "obtain if it uses the given policy. To obtain the return, we predict the value function\n",
    "or Q function using the given policy.\n",
    "\n",
    "That is, we learned that the value function or value of a state denotes the expected\n",
    "return an agent would obtain starting from that state following some policy π. Thus,\n",
    "by predicting the value function using the given policy π, we can understand what\n",
    "the expected return the agent would obtain in each state if it uses the given\n",
    "policy π. If the return is good then we can say that the given policy is good.\n",
    "\n",
    "Similarly, we learned that the Q function or Q value denotes the expected return the\n",
    "agent would obtain starting from the state s and an action a following the policy π .\n",
    "Thus, predicting the Q function using the given policy π, we can understand what the\n",
    "expected return the agent would obtain in each state-action pair if it uses the given\n",
    "policy. If the return is good then we can say that the given policy is good.\n",
    "\n",
    "Thus, we can evaluate the given policy π by computing the value and Q functions.\n",
    "Note that, in the prediction task, we don't make any change to the given input policy.\n",
    "We keep the given policy as fixed and predict the value function or Q function using\n",
    "the given policy and obtain the expected return. Based on the expected return, we\n",
    "evaluate the given policy.\n",
    "\n",
    "\n",
    "## Control task\n",
    "\n",
    "Unlike the prediction task, in the control task, we will not be given any policy as\n",
    "an input. In the control task, our goal is to find the optimal policy. So, we will start\n",
    "off by initializing a random policy and we try to find the optimal policy iteratively.\n",
    "That is, we try to find an optimal policy that gives the maximum return.\n",
    "\n",
    "Thus, in a nutshell, in the prediction task, we evaluate the given input policy by\n",
    "predicting the value function or Q function, which helps us to understand the\n",
    "expected return an agent would get if it uses the given policy, while in the control\n",
    "task our goal is to find the optimal policy and we will not be given any policy as\n",
    "input; so we will start off by initializing a random policy and we try to find the\n",
    "optimal policy iteratively.\n",
    "\n",
    "Now that we have understood what prediction and control tasks are, in the next\n",
    "section, we will learn how to use the Monte Carlo method for performing the\n",
    "prediction and control tasks."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
